Black Box Research Articles

Global and local interpretability techniques of supervised machine learning black box models for numerical medical data

The most effective machine learning classification techniques, such as artificial neural networks, are not easily interpretable, which limits their usefulness in critical areas, such as medicine, where errors can have severe consequences. Researchers have been working to balance the trade-off between the model performance and interpretability. In this study, seven interpretability techniques (global surrogate, accumulated local effects, local interpretable model-agnostic explanations (LIME), Shapley additive explanations (SHAP), model agnostic post hoc local explanations (MAPLE), local rule-based explanation (LORE), and Contextual Importance and Utility (CIU)) were evaluated to interpret five medical classifiers (multilayer perceptron, support vector machines, random forests, extreme gradient boosting, and naïve bayes) using six model performance metrics and three interpretability technique metrics across six medical numerical datasets. The results confirmed the effectiveness of integrating global and local interpretability techniques, and highlighted the superior performance of global SHAP explainer and local CIU explanations. The quantitative evaluations of explanations emphasised the importance of assessing these interpretability techniques before employing them to interpret black box models.

Physics-informed neural networks in groundwater flow modeling: Advantages and future directions

In recent years, there has been enormous development in soft computing, especially artificial intelligence (AI), which has developed robust methods for solving complex engineering problems. Researchers in the field of water resources engineering have applied these AI methods to solve a variety of hydrological problems. Despite their widespread use in the surface and atmospheric hydrology fields, groundwater hydrologists have not widely used AI methods in their routine field-scale modeling efforts. This is because AI models have been primarily considered black box models that lack physical meaning. Furthermore, using AI models to generate the space-time distribution of transient groundwater level variations is challenging and requires further flux balance and mass transport analyses. More recently, a new type of physics-informed neural network (PINN) model has been developed to address several limitations by integrating governing physics (groundwater flow equations) into the AI tools. This study presents the systematic advantages of the PINN algorithm for solving groundwater problems using a set of classic test problems. As discussed in detail in the article, these advantages and potentials are associated with the meshless nature of PINN, its continuous time and space dimensions, its independence from time-stepping and incremental marching in space, and its efficiency in running time. However, despite PINN's promising attributes, it is important to acknowledge its nascent stage of development and the inherent limitations of all neural network models, such as training challenges and hyperparameter selection. Thus, collaborative efforts between groundwater modelers and computer scientists are imperative to explore and exploit the full potential of PINN in tackling increasingly complex groundwater problems and nurturing PINN into a dependable modeling tool in industry and academia.

Exploring a link between faculty intrapreneurship, student entrepreneurship and ecosystem dynamism

Purpose This study aims to understand the impact of student entrepreneurship and university support on faculty intrapreneurship. The authors also analyze the role of the university’s dynamic and ordinary capabilities and the environmental dynamism in which the university is embedded. Design/methodology/approach With a large survey data set involving 680 professors and 2,230 students from 70 Brazilian universities, the authors use a multimethod approach with partial least squares structural equation modeling (PLS-SEM) and fuzzy-set qualitative comparative analysis (fsQCA). Findings The PLS-SEM results demonstrate that student entrepreneurship indirectly influences faculty intrapreneurship through the interaction of students with faculty and entrepreneurs, in addition to proving the intense influence of university support on faculty intrapreneurship, especially in a slow-growth environment. Additionally, the authors confirmed the moderating effect of universities’ dynamic and ordinary capabilities on student interaction and university support, respectively, and some exciting differences considering the ecosystem dynamism. The fsQCA results deepened the differences between environments, presenting different configurations between the antecedents that lead to high levels of faculty intrapreneurship in fast and slow-growth environments. Originality/value The study makes a unique and significant contribution to the literature on faculty intrapreneurship by examining the cross-interactions between individual, organizational and environmental levels about the promotion of faculty intrapreneurship. From a practical point of view, it is possible to identify more effective, innovative and systematic ways to encourage faculty intrapreneurship in a developing country. The findings help open up the black box of faculty intrapreneurship.

Making the black box more transparent: improving the reporting of artificial intelligence studies in healthcare

Making the black box more transparent: improving the reporting of artificial intelligence studies in healthcare

Satisfaction with human resource practices, job dedication and job performance: the role of incentive gamification

PurposeWhile the potential of human resource practices (HRPs) for promoting performance is widely recognized, even though crucial, employees’ assessment of HRPs remains under-researched, especially in emerging economies. Hence, the purpose of this research is to examine the influence of employee satisfaction with HRPs on job performance through the mediating role of job dedication (JD) and the moderating role of incentive gamification.Design/methodology/approachThe current research adopts a quantitative method. Specifically, using a questionnaire survey, 418 valid responses collected (through purposive sampling) via cross-sectional method from the employees of Nigerian Information and Communication Technology (ICT) firms were used to test the research hypotheses empirically.FindingsThe results revealed that satisfaction with HRPs has a positive influence on job performance. Satisfaction with human resource practices has a positive influence on job dedication. Job dedication has a positive influence on job performance. The link between employees’ satisfaction with human resource practices and job performance is mediated by job dedication. The link between satisfaction with human resource practices and job dedication is moderated by incentive gamification, that is, the positive link is stronger when incentive gamification is high. The link between satisfaction with human resource practices and job performance is moderated by incentive gamification, that is, the positive link is stronger when incentive gamification is high.Originality/valueThe current study highlights the importance of employees’ assessments of human resource practices, which may be used to promote employee dedication, which in turn results in improved performance. The findings are helpful to open the black box of the impact of satisfaction with HRPs on job performance. The results also offer important new valuable guidance for practitioners and will aid the management of human resource practice design, audits, and communication.

Visual explanations of machine learning model estimating charge states in quantum dots

Charge state recognition in quantum dot devices is important in the preparation of quantum bits for quantum information processing. Toward auto-tuning of larger-scale quantum devices, automatic charge state recognition by machine learning has been demonstrated. For further development of this technology, an understanding of the operation of the machine learning model, which is usually a black box, will be useful. In this study, we analyze the explainability of the machine learning model estimating charge states in quantum dots by gradient weighted class activation mapping. This technique highlights the important regions in the image for predicting the class. The model predicts the state based on the change transition lines, indicating that human-like recognition is realized. We also demonstrate improvements of the model by utilizing feedback from the mapping results. Due to the simplicity of our simulation and pre-processing methods, our approach offers scalability without significant additional simulation costs, demonstrating its suitability for future quantum dot system expansions.

The KGB’s Black Box: Intelligence Operations and Bias in the People’s Republic of China

The Soviet Union's intelligence operations in China helped to fuel tensions between the two communist powers, which were based on analytical biases and the difficulty of operating in China. While the Soviet Union was worried about a potential Chinese attack, it found it hard to penetrate Chinese political circles due to cultural differences and difficulties in recruiting Chinese spies. Prior to the Cold War, Soviet intelligence agencies had an easier time recruiting Chinese nationals as spies, but as the Sino-Soviet split deepened, recruiting agents, penetrating political circles, and making sense of China became more challenging. The Soviet Union's biased analytical framework on China contributed to escalatory tensions and created a feedback loop of distrust. While there are limitations to analyzing leaked USSR official notes carried by a defector, they provide valuable insights into Soviet operations in China. The KGB established its "Line K" department for Chinese espionage and operations, but complained about a chronic shortage of Chinese documents passed along by assets that would enable it to draw serious assessments about the Chinese communist government.

In Situ Imaging of Two-Dimensional Crystal Growth Using a Heat-Resistant Optical Microscope.

Revealing low-dimensional material growth dynamics is critical for crystal growth engineering. However, in a practical high-temperature growth system, the crystal growth process is a black box because of the lack of heat-resistant imaging tools. Here, we develop a heat-resistant optical microscope and embed it in a chemical vapor deposition (CVD) system to investigate two-dimensional (2D) crystal growth dynamics. This in situ optical imaging CVD system can tolerate temperatures of ≤900 °C with a spatial resolution of ∼1 μm. The growth of monolayer MoS2 crystals was studied as a model for 2D crystal growth. The nucleation and growth process have been imaged. Model analysis and simulation have revealed the growth rate, diffusion coefficient, and spatial distribution of the precursor. More importantly, a new vertex-kink-ledge model has been suggested for monolayer crystal growth. This work provides a new technique for in situ microscopic imaging at high temperatures and fundamental insight into 2D crystal growth.

Explainable AI-driven model for gastrointestinal cancer classification

Although the detection procedure has been shown to be highly effective, there are several obstacles to overcome in the usage of AI-assisted cancer cell detection in clinical settings. These issues stem mostly from the failure to identify the underlying processes. Because AI-assisted diagnosis does not offer a clear decision-making process, doctors are dubious about it. In this instance, the advent of Explainable Artificial Intelligence (XAI), which offers explanations for prediction models, solves the AI black box issue. The SHapley Additive exPlanations (SHAP) approach, which results in the interpretation of model predictions, is the main emphasis of this work. The intermediate layer in this study was a hybrid model made up of three Convolutional Neural Networks (CNNs) (InceptionV3, InceptionResNetV2, and VGG16) that combined their predictions. The KvasirV2 dataset, which comprises pathological symptoms associated to cancer, was used to train the model. Our combined model yielded an accuracy of 93.17% and an F1 score of 97%. After training the combined model, we use SHAP to analyze images from these three groups to provide an explanation of the decision that affects the model prediction.

Interpretable AI Services for Enhanced Air Quality Forecasting

Most of the Machine Learning (ML) models used these days establish a complex relationship between the in- dependent variables (X) and dependent variable (y). Without understanding the relationship, we risk introducing undesirable features into the predictions. Biased collection of the data, used to build the model, might bolster these undesirable features. The model might soon become unfit for its intended tasks. This project tries to get deeper insights into such black box machine learning models by looking into various ExplainableAI (XAI) tools and provide it as a service to users. These tools when used in conjunction can make complex models easy to understand and operate for the end-user. Specifically, the tools used would help the user of the machine learning model interact with it and monitor how it behaves on changing certain aspects of the data. To facilitate the better understanding of the achieved outcome, this project uses a weather data-set which is used to classify the air quality.

Unpacking the black box: Investigating the role of social protection programmes in promoting decent work and economic growth in low‐income countries

AbstractThis study investigates the intricate nexus between low‐income countries, Sustainable Development Goals (SDGs), and the multifaceted phenomenon of unemployment. Amid the global endeavour to achieve SDGs, these marginalised nations assume a pivotal role, confronting distinctive challenges and opportunities. We employ meticulous analysis and structural equation modelling techniques to unveil a complex network of relationships among diverse socio‐economic indicators and unemployment across 27 low‐income countries, encompassing nations such as Afghanistan, Zambia, and others. Our empirical findings underscore the paramount importance of economic advancement, as epitomised by GDP per capita, in fostering employment prospects. Concurrently, environmental endeavours, epitomised by CO2 emissions, underscore the potential of green employment in mitigating unemployment. Investments in education and healthcare, as indicated by primary completion rates and life expectancy, emerge as pivotal determinants of employability. Moreover, access to clean energy sources and mobile cellular subscriptions indirectly influence unemployment rates, underscoring the multifaceted nature of the issue. Furthermore, our analysis reveals the intricate interplay between economic progress and social well‐being. These insights give policymakers invaluable direction, stressing the significance of tailored strategies that harness education, healthcare, technology, and environmental initiatives to alleviate unemployment, cultivate inclusive growth, and advance the overarching aspirations of sustainable development in low‐income countries.

AMBER Alert: An Ironic Antidote to “Missing White Girl Syndrome”?

“Missing White Girl Syndrome” (MWGS) has been coined in critique of news media coverage favoritism of victims fitting that description. The AMBER Alert child recovery system was in fact inspired by the abduction and murder of a missing white girl, and some have argued its issuance decisions reflect MWGS. We examine this question using secondary data, including the annual AMBER Alert reports provided by the National Center for Missing and Exploited Children (NCMEC), the National Incidence Studies of Missing, Abducted, Runaway, and Thrownaway Children (NISMART), and the National Crime Information Center (NCIS). We compared the demographics of victims from official AMBER Alert data (NCMEC) with cross-comparable demographics from the two official missing persons data sources to test for potential main-effect bias favoring whites or girls. A fourth dataset derived from media accounts of AMBER Alerts is used to test for any potential interaction effect of issuance preference for white girls. Overwhelmingly the findings suggest AMBER Alert issuance decisions do not reflect MWGS. Thus, we argue the system has the potential to function as an ironic “antidote” to MWGS by virtue of the identities of the children for whom it is routinely deployed. However, we qualify by acknowledging limitations with available data, and that the “black box” of AMBER Alert issuance decisions remains unstudied, leaving open the question of race and/or gender bias occurring at that point. Policy discourse and future research implications are discussed.

Explainable and visualizable machine learning models to predict biochemical recurrence of prostate cancer.

Machine learning (ML) models presented an excellent performance in the prognosis prediction. However, the black box characteristic of ML models limited the clinical applications. Here, we aimed to establish explainable and visualizable ML models to predict biochemical recurrence (BCR) of prostate cancer (PCa). A total of 647 PCa patients were retrospectively evaluated. Clinical parameters were identified using LASSO regression. Then, cohort was split into training and validation datasets with a ratio of 0.75:0.25 andBCR-related features were included in Cox regression and five ML algorithm to construct BCR prediction models. The clinical utility of each model wasevaluated by concordance index (C-index) values and decision curve analyses (DCA). Besides, Shapley Additive Explanation (SHAP) values were used to explain the features in the models. We identified 11 BCR-related features using LASSO regression, then establishing five ML-based models, including random survival forest (RSF), survival support vector machine (SSVM), survival Tree (sTree), gradient boosting decision tree (GBDT), extreme gradient boosting (XGBoost), and a Cox regression model, C-index were 0.846 (95%CI 0.796-0.894), 0.774 (95%CI 0.712-0.834), 0.757 (95%CI 0.694-0.818), 0.820 (95%CI 0.765-0.869), 0.793 (95%CI 0.735-0.852), and 0.807 (95%CI 0.753-0.858), respectively. The DCA showed that RSF model had significant advantages over all models. In interpretability of ML models, the SHAP value demonstrated the tangible contribution of each feature in RSF model. Our score system provide reference for the identification for BCR, and the crafting of a framework for making therapeutic decisions for PCa on a personalized basis.

Weighted Bayesian Belief Network for diabetics: a predictive model

Diabetes is an enduring metabolic condition identified by heightened blood sugar levels stemming from insufficient production of insulin or ineffective utilization of insulin within the body. India is commonly labeled as the “diabetes capital of the world” owing to the widespread prevalence of this condition. To the best of the authors' last knowledge updated on September 2021, approximately 77 million adults in India were reported to be affected by diabetes, reported by the International Diabetes Federation. Owing to the concealed early symptoms, numerous diabetic patients go undiagnosed, leading to delayed treatment. While Computational Intelligence approaches have been utilized to improve the prediction rate, a significant portion of these methods lacks interpretability, primarily due to their inherent black box nature. Rule extraction is frequently utilized to elucidate the opaque nature inherent in machine learning algorithms. Moreover, to resolve the black box nature, a method for extracting strong rules based on Weighted Bayesian Association Rule Mining is used so that the extracted rules to diagnose any disease such as diabetes can be very transparent and easily analyzed by the clinical experts, enhancing the interpretability. The WBBN model is constructed utilizing the UCI machine learning repository, demonstrating a performance accuracy of 95.8%.

Interdependent Formation of Symbolic and Regulatory Boundaries: The Discursive Contestation Around the Home-Sharing Category

The formation of boundaries between established and emergent categories is a complex social process. Therein, our understanding of how symbolic boundaries translate into regulatory boundaries is underdeveloped. Extant research either treats laws and regulations for categories as given or assumes a seamless translation of a symbolic into a regulatory boundary. This sidelines that market participants actively contest and shape boundaries between categories. To address this lacuna, we open the black box of how symbolic boundaries are translated into regulatory boundaries. We adopt a discursive perspective and conduct a longitudinal study of the contestation around the categories of home sharing and short-term rental in Europe. Our analysis shows how symbolic and regulatory boundaries are formed in a causal sequential process, driven by shifts in the field positioning of market actors and in the discursive accounts they mobilize. We develop a theoretical model of the discursive foundation of category boundary formation. At the heart of our theorization are discursive accounts and how shifting coalitions of market participants mobilize them to shape the evolving symbolic and regulatory boundaries between an emergent and an established category. We contribute to category research by unearthing the interdependent formation of symbolic and regulatory boundaries and the role of discursive accounts in these processes.

More than the SRY: The Non-Coding Landscape of the Y Chromosome and Its Importance in Human Disease

Historically, the Y chromosome has presented challenges to classical methodology and philosophy of understanding the differences between males and females. A genetic unsolved puzzle, the Y chromosome was the last chromosome to be fully sequenced. With the advent of the Human Genome Project came a realization that the human genome is more than just genes encoding proteins, and an entire universe of RNA was discovered. This dark matter of biology and the black box surrounding the Y chromosome have collided over the last few years, as increasing numbers of non-coding RNAs have been identified across the length of the Y chromosome, many of which have played significant roles in disease. In this review, we will uncover what is known about the connections between the Y chromosome and the non-coding RNA universe that originates from it, particularly as it relates to long non-coding RNAs, microRNAs and circular RNAs.

Improving traffic accident severity prediction using MobileNet transfer learning model and SHAP XAI technique.

Traffic accidents remain a leading cause of fatalities, injuries, and significant disruptions on highways. Comprehending the contributing factors to these occurrences is paramount in enhancing safety on road networks. Recent studies have demonstrated the utility of predictive modeling in gaining insights into the factors that precipitate accidents. However, there has been a dearth of focus on explaining the inner workings of complex machine learning and deep learning models and the manner in which various features influence accident prediction models. As a result, there is a risk that these models may be seen as black boxes, and their findings may not be fully trusted by stakeholders. The main objective of this study is to create predictive models using various transfer learning techniques and to provide insights into the most impactful factors using Shapley values. To predict the severity of injuries in accidents, Multilayer Perceptron (MLP), Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), Residual Networks (ResNet), EfficientNetB4, InceptionV3, Extreme Inception (Xception), and MobileNet are employed. Among the models, the MobileNet showed the highest results with 98.17% accuracy. Additionally, by understanding how different features affect accident prediction models, researchers can gain a deeper understanding of the factors that contribute to accidents and develop more effective interventions to prevent them.

Artificial Intelligence, Lymphoid Neoplasms, and Prediction of MYC, BCL2, and BCL6 Gene Expression Using a Pan-Cancer Panel in Diffuse Large B-Cell Lymphoma

Background: Artificial intelligence in medicine is a field that is rapidly evolving. Machine learning and deep learning are used to improve disease identification and diagnosis, personalize disease treatment, analyze medical images, evaluate clinical trials, and speed drug development. Methods: First, relevant aspects of AI are revised in a comprehensive manner, including the classification of hematopoietic neoplasms, types of AI, applications in medicine and hematological neoplasia, generative pre-trained transformers (GPTs), and the architecture and interpretation of feedforward neural net-works (multilayer perceptron). Second, a series of 233 diffuse large B-cell lymphoma (DLBCL) patients treated with rituximab-CHOP from the Lymphoma/Leukemia Molecular Profiling Project (LLMPP) was analyzed. Results: Using conventional statistics, the high expression of MYC and BCL2 was associated with poor survival, but high BCL6 was associated with a favorable overall survival of the patients. Then, a neural network predicted MYC, BCL2, and BCL6 with high accuracy using a pan-cancer panel of 758 genes of immuno-oncology and translational research that includes clinically relevant actionable genes and pathways. A comparable analysis was performed using gene set enrichment analysis (GSEA). Conclusions: The mathematical way in which neural networks reach conclusions has been considered a black box, but a careful understanding and evaluation of the architectural design allows us to interpret the results logically. In diffuse large B-cell lymphoma, neural networks are a plausible data analysis approach.

Prediction and Elucidation of Physical Properties of Polycrystalline Materials Using Multichannel Machine Learning of Electron Backscattering Diffraction

AbstractThe application of machine learning in materials science has yielded several benefits, including the prediction of physical properties and the improvement of experimental efficiency. However, with complex models, such as convolutional neural networks (CNN), learning has become a black box, from which no universal physical knowledge can be obtained. In this study, a highly accurate prediction of the electrical properties of polycrystalline semiconductor thin films is achieved by learning multichannel CNN models from electron backscattering diffraction (EBSD) data, that is band contrasts, grain boundaries, and inverse pole figures. In addition, it examines how the CNN model learned the correlation between the crystallinity, grain boundaries, crystallographic orientation, and carrier mobility by polarizing certain EBSD data and checking the predicted changes in carrier mobility. Physical parameters affecting carrier mobility can be extracted, which is challenging via human image recognition. The methods proposed in this study will not only enable the prediction of electrical properties from EBSD data for all materials but also will contribute to the discovery of complex physical phenomena beyond the limits of human analysis.

A Meta Algorithm for Interpretable Ensemble Learning: The League of Experts

Background. The importance of explainable artificial intelligence and machine learning (XAI/XML) is increasingly being recognized, aiming to understand how information contributes to decisions, the method’s bias, or sensitivity to data pathologies. Efforts are often directed to post hoc explanations of black box models. These approaches add additional sources for errors without resolving their shortcomings. Less effort is directed into the design of intrinsically interpretable approaches. Methods. We introduce an intrinsically interpretable methodology motivated by ensemble learning: the League of Experts (LoE) model. We establish the theoretical framework first and then deduce a modular meta algorithm. In our description, we focus primarily on classification problems. However, LoE applies equally to regression problems. Specific to classification problems, we employ classical decision trees as classifier ensembles as a particular instance. This choice facilitates the derivation of human-understandable decision rules for the underlying classification problem, which results in a derived rule learning system denoted as RuleLoE. Results. In addition to 12 KEEL classification datasets, we employ two standard datasets from particularly relevant domains—medicine and finance—to illustrate the LoE algorithm. The performance of LoE with respect to its accuracy and rule coverage is comparable to common state-of-the-art classification methods. Moreover, LoE delivers a clearly understandable set of decision rules with adjustable complexity, describing the classification problem. Conclusions. LoE is a reliable method for classification and regression problems with an accuracy that seems to be appropriate for situations in which underlying causalities are in the center of interest rather than just accurate predictions or classifications.